System and method for ergonomic tracking for individual physical exertion

ABSTRACT

A system and method are provided for tracking a user&#39;s posture. The system and method include an environmental module for determining the user&#39;s physical environment, a biomechanical module for determining at least a user&#39;s posture in the user&#39;s physical environment, and an output module for outputting to the user an indication of at least the user&#39;s posture relative to at least a target correct posture. The physical environment can include a computer workstation environment, a manufacturing environment, a gaming environment, and/or a keypadding environment.

RELATED APPLICATION

This application claims the benefit of U.S. Provisional Application No.60/576,726, filed on Jun. 3, 2004, the entire teachings of which areherein incorporated by reference.

BACKGROUND OF THE INVENTION

In ergonomics, physical exertion (e.g. work) is studied to try to reduceuser fatigue and discomfort. As an individual exerts physical effort,physiological and biomechanical factors interweave. Physiologically, thehuman structures of ergonomic issue are skeletal and muscular.Individuals of differing shoulder breadth, for example, will positionsomewhat differently over a same-size keyboard. Biomechanically,components of ergonomic issue include posture, force, repetition andvibration. An upright sitting posture while manipulating a pointingdevice, for example, engenders different body exertion than a slouchingposture.

Posture ranks right up at the top of the list when you are talking aboutgood health. It is as important as eating right, exercising, getting agood night's sleep and avoiding potentially harmful substances likealcohol, drugs and tobacco. Good posture is a way of doing things withmore energy, less stress and fatigue. Without good posture, a person'soverall health and total efficiency may be compromised. Because thelong-term effects of poor posture can affect bodily systems (such asdigestion, elimination, breathing, muscles, joints and ligaments), aperson who has poor posture may often be tired or unable to workefficiently or move properly.

Poor posture could bring on more severe musculoskeletal disorders (MSDs)such as ruptured disc or carpal tunnel syndrome. Excessive loading ofthe back musculoskeletal structures could weaken and even rupture aspinal disc. Carpal tunnel syndrome is normally caused by repetitive useof a hand or a wrist, where posture of the larger musculoskeletalstructures (neck, arm) and/or finer structures (wrist, fingers) affectsloading.

MSDs can happen to anyone who exerts repeated physical effort overperiods of time. Stressful wrist, arm, neck and/or back positions,whether from working at a desk, long distance driving or lifting boxes,only aggravate the potential for damage.

SUMMARY OF THE INVENTION

The present invention provides a low cost non-invasive mechanism forpreventing various kinds of incapacitating trauma that occur throughincorrect ergonomic usage. The present invention uses real-timemirroring and positive modeling to address both prevention andintervention purposes.

A system and method are provided for tracking a user's posture. Thesystem and method include an environmental module for determining theuser's physical environment, a biomechanical module for determining atleast a user's posture in the user's physical environment, and an outputmodule for outputting to the user an indication of at least the user'sposture relative to at least a target correct posture. The physicalenvironment can include a computer workstation environment, amanufacturing environment, a gaming environment, and/or a keypaddingenvironment.

In a preferred embodiment, the environmental module includes a scenemodule and a snapshot module, the scene and the snapshot modules capturea digital representation of a background of the user's physicalenvironment. The biomechanical module includes an active scene modulefor capturing a digital representation of the user in the user'sphysical environment, and a runtime mirror module for processing thedigital representation of the user in the user's physical environment.The output module includes a display for displaying to the user theuser's posture relative to a target correct posture.

In one embodiment, the output module further includes an audio devicefor outputting to the user an audio indication of the user's posturerelative to the target correct posture. The output can be displayed in awindow of a multi-window environment. The representation of the user'sposture can be superimposed with the target correct posture.

In one embodiment at least one digital camera can be used in determiningthe user's physical environment and/or in determining the user'sposture.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features and advantages of theinvention will be apparent from the following more particulardescription of preferred embodiments of the invention, as illustrated inthe accompanying drawings in which like reference characters refer tothe same parts throughout the different views. The drawings are notnecessarily to scale, emphasis instead being placed upon illustratingthe principles of the invention.

FIG. 1 illustrates a correct target posture and associated ergonomicmetrics;

FIG. 2 is a schematic view of a tracking system during keyboardingactivities;

FIG. 3 is a flow diagram of the present invention;

FIG. 4 is a flow diagram of an optional process in addition to the flowdiagram described in FIG. 3; and

FIGS. 5A-5C illustrate a user in a repetitive motion-type environment.

DETAILED DESCRIPTION OF THE INVENTION

A description of preferred embodiments of the invention follows.

Ergonomic factors are expressed with respect to an environment ofphysical exertion. That is, the particular expression of variousergonomic factors yields ergonomic metrics pertaining to a particulartype of physical environment, e.g. clerical, manufacturing. The physicalenvironment includes an individual (user) and a set of needed tools,e.g. computing keyboard, pointing device. Thus, the set of ergonomicmetrics express the recommended position of the working individual withrespect to his/her type of tools.

Target correct posture guidelines for computer workstation usage havebeen established by ANSI/HFS 100-1988 (ANSI) and BSR/HFES 100 DraftStandard, the entire teachings of which are herein incorporated byreference. FIG. 1 illustrates a correct target posture and associatedergonomic metrics as described in the aforementioned guidelines.

The published national guidelines include seat pan, work surface andscreen viewing heights (h2, h3, and h1 respectively) to the nearesttenth-inch/centimeter (ANSI) or alternatively elbow, shoulder abductionand flexion, wrist flexion and extension, and torso-to-thigh angles(BSR) to the nearest degree. The guideline measurements, if compliedwith, dictate that the individual's posture must correspond to within atenth-inch/centimeter or one angle degree with respect to posturesegments or angles (such as distance “d” and angle “a” in FIG. 1).

An instrumentation approach has been used in specialized cases toprovide a user ergonomic metrics based upon the user's work environment.To achieve sufficient accuracy, the instrumentation approach typicallyincludes devices such as goniometers and/or height yardsticks, requiringphysically-proximate equipment around or next to each computing user.When a user changes from a seated position to a standing position, orjust reaches for a cup of coffee, the instrumentation approach toachieve accuracy would require repositioning all/some equipment in orderto sample all measurements again. The aforementioned approach has notbeen practical to implement apart from controlled settings.

Another implementation of the instrumentation approach can incorporatewired or wireless body sensors attached at various points on the user'sbody. This approach would require such sensors to be attached daily, andto be unattached again when leaving the work environment. Again, theinstrumentation approach has not been practical to implement apart fromcontrolled settings.

Standard approaches today use written checklists or a one-time expertevaluation, so that an individual may try to internalize the 5-10measurement metrics to continuously apply.

The present invention utilizes an alternative vision-based approachwhich requires no equipment physically around or next to eachindividual. The scope of user motion, such as seated/standing orreaching for coffee, would increase without requiring such repositioningas needed in the instrumentation approach. These user benefits couldallow for more widescale adoption of ergonomic practices, by minimizinginvasiveness and individual adoption cost.

FIG. 2 shows an ergonomic tracking system 100 that is used in oneembodiment, to help a user 110 to have correct posture duringkeyboarding activities. In general, a camera 140 observes a user's 110physical environment, such as the user entering information at acomputer station or input area. The computer station or input area mayinclude, alone or in combination, a chair 120, a work surface 122, acentral processing unit 130, a display monitor 132, a keyboard 134, amouse 136, or other devices. Other devices can be for example, a keypad,a joystick, a trackball, a touchpad, a wand, a touchscreen, a printer orany other known input or output device. Based on the user's 110 physicalenvironment, the system 100 determines the user's 110 posture andoutputs to the user 110 an indication of the user's posture relative toa target correct posture.

In a particular embodiment, the user 110 at the keyboard 134 or otherinput device is viewed using one camera 140. Digital images generated bythe camera 140 or associated third party image software are processed bythe system 100 to detect the body, the keyboard 134 (or other inputdevice), the display monitor 132 and other aspects of the userenvironment, in each frame. In one embodiment, the system 100 or camera140 can upload the digital images to the Internet for viewing or furtherprocessing. The camera can also be an embedded camera (e.g. in computer130, cell phone or other device). It should be understood that thesystem 100 may be implemented with multiple cameras 140.

FIG. 3 shows a flow diagram 200 of an implementation of the ergonomictracking system 100 of FIG. 2. In one embodiment, the system 100includes an initialization module 210, a setup module 220, an activescene module 240, a first determination module 250, a runtime mirrormodule 260, a display module 270, a second determination module 280, anda third determination module 290.

Initialization Module (210) allows the user 110 to initialize or “start”the ergonomic tracking system 100 of FIG. 2. This can be accomplished bythe user 110 “clicking” or otherwise selecting an icon on the monitor132 screen view using the mouse 136 or other input device. The iconactivates the ergonomic tracking system 100.

The setup module 220 includes a scene template module 222 and a snapshotmodule 224. The scene template module 222 and the snapshot module 224capture the background of the user's environment without the user beingpresent in the scene. The physical environment can include (a) thekeyboard 134 and/or other input device(s); (b) the viewingscreen/monitor/embedded display 132; (c) the chair 120, adesk/pedestal/other supporting furniture 122; and (d) any other elementsin the physical environment. The system 100 takes an initial picture orsnapshot with digital camera 140. The camera 140 can be either operatedautomatically via electronic self-timer, “movie mode” or other existingcamera means, or by a third party. The setup module 220 uses resultingdigital image from the camera 140 to form the initial background sceneof the user's environment. In one embodiment, the setup module onlyneeds to be run once unless the environment materially changes, e.g. thefurniture was reconfigured.

The active scene module 240 allows the user 110 to physically enter the“scene” and assumes his/her generally working (inputting) position. Forexample, the user 110 can sit at the computer station and positionhis/her hands over the keyboard 134 or the mouse 136 of FIG. 2.

Intermediate human representation is comprised of an angled vector spaceand/or other internal data model. The active scene module 240anticipates successive tracking in nearby representational space. Assuch, modeling of the user 110 and session-level information may bestored for this user. Application-level information may be stored for auser, company (or a home environment). Further, modeling information canbe sent (anonymously) to a central repository such that modeling can beimproved for subsequent users.

The first determination module 250 allows the user 110 to decide whetherto start tracking (252) or remain in a “wait state” (254). If the user110 decides to start tracking (252), the user 110 clicks an icon on themonitor 132 screen view using the mouse 136 or other input means. Onceviewing has started, the system 200 takes a snapshot of the “scene” withthe user 110 present using the digital camera 140.

To model the user's body at the keyboard 134 or other input device, thedigital images from the camera 140 are processed to detect the user 110in the physical environment and retain an intermediate representation ofthe user 110 for successive image processing and display processing. Forexample, an approximate profile view may be sufficient for obtaining arepresentative pose.

As shown with reference to FIG. 2, ergonomic positions generally involveas much body bilateral symmetry as feasible. Further, using an inputdevice commonly involves activation of large muscles (e.g. spinal, arms,legs) and fine hand/finger muscles, with upper limbs ergonomicallyproximate to the body. Therefore, sufficient pose information can begleaned from 2-D digital images. However, although a single camera 140is used in keeping costs low, there may be occasions where one cameraview does not yield sufficient information. As such, multiple cameras140 providing multiple camera views may be used to produce for example afrontal profile and/or an oblique view.

The runtime mirror module 260 processes the snapshot of the user's 110image. The processed snapshot yields/derives a representation of theuser's 110 posture (position) in the scene. Multiple independentdetector, segmenter, estimator, and/or classifier modules can be used toachieve accuracy in real-time with low error rates. As the user 110inevitably changes working (input) position the user's 110 movements aretracked by runtime mirror module 260. Mirroring reflects such physicalchanges to the positions of the user's 110 various body parts. Mirroringincludes both body modeling and body tracking.

Mirroring of the user's 110 body at the keyboard 134 or other inputdevice occurs in real-time. Thus, tracking of large user movementsand/or fine user movements keeps pace over time. For example, largemovements such as standing up or sitting down can involve shifting fromone major ergonomically-sanctioned position to another.

The display module 270 outputs the processed snapshot to the user 110 ina screen view rendered or otherwise displayed on the monitor 132 in anapplication window. The application window can be one of multiplewindows running in a windowing system (such as Microsoft Windows).

In one embodiment, the screen view includes the user's image in theenvironment of the scene (i.e. current position), superimposed by orotherwise in respect to a target correct posture indication. The targetcorrect posture indication may be produced by line art, graphics,shading techniques (with or without the user's image), and the like. Thesystem 200 may also provide visual and/or audio cues to help the user110 achieve a correct target posture.

Use of the display module 270 allows a user 110 to learn/train fromhis/her previous activity. As for many kinds of physical activity, theuser 110 learns to correctly position his/her body parts using suchphysical modeling. Both negative modeling (incorrect pose) and positivemodeling (target pose or “how-to-get-to” target pose) contribute to thephysicality of user 110 learning, such that “muscle memory” engages. Ifa pose is incorrect, visual and/or audio cues signal the user 110. Thesevisual and/or audio cues can be a graphic highlight/color, an audiosignal, a flashing signal, and/or other visual and audio effects known.

In some embodiments, viewing may be used more than training. Forexample, depending upon the level of a user's 110 health and corporatepolicy, a user 110 may opt to run viewing in the windowing foreground atselect points during the day or simply in the background all day long.

The second and third determination modules 280, 290 allow the user 110to determine whether to suspend viewing (282), continue viewing (284,294) or quit the system (292). If the user 110 decides to continueviewing (284, 294), the system 200 takes another snapshot of the “scene”at 252, and repeats modules 260, 270, 280 and 290 until the user 110decides to quit (292) or suspend (282) the system 100.

FIG. 4 shows a flow diagram 300 of an optional training process inaddition to the flow diagram described in FIG. 3. In another embodiment,the system 100 (FIG. 2) can include a display gallery module 310, amirror selection module 320, and a setup determination module 330.

The display gallery module 310 allows user 110 (FIG. 2) to cycle througha display of working postures. For example, ANSI-approved seating andstanding poses. These serve as candidate target correct postures forvarious poses.

The mirror selection module 320 allows the user 110 to select one of theworking postures provided by the display in 310. The chosen workingposture is compared to the user's 110 actual (camera 140 imaged)posture.

The setup determination module 330 prompts the user 110 whether thesetup module 220 (FIG. 3) has been completed. If the setup module 220has been completed (332), the system 300 proceeds with the runtimemirroring module 260 described above in FIG. 2. If the setup module 220has not been completed (334), the system 300 proceeds to the setupmodule 220.

The present invention provides a low cost mechanism for preventingvarious kinds of incapacitating repetitive trauma that occur throughincorrect ergonomic usage. The present invention uses real-timemirroring and positive modeling to address both prevention andtherapeutic purposes.

FIGS. 5A-5C illustrate a user in a repetitive motion-type environment.The user 400 reaches for items 402 coming off a conveyer belt 404. Theuser 400 then places the items 402 in a packaging container 406. Oncethe user 400 fills the packaging container with items 402, the user 400closes the packaging container 406. The embodiments of the presentinvention can be used to provide the user 400 with correct bodypositioning information with relation to this type of repetitive motion(i.e. throughout the series of body positions forming the repetitivemotion). As such, traumas such as lower back injury may be avoided.

Other embodiments of the invention can be used for keyboarding, gaming,keypadding and the like. Keyboard users commonly input via a variety ofalternative keyboards and pointing devices, generally while viewing on a(screen) monitor associated with a computer, web TV or networked systemsuch as the Internet. Use of other keyboards however may be included,such as PDAs, handheld electronic devices, portable phones, or textmessaging systems, etc. Gaming users input via a variety of keyboardsand/or embedded joysticks, triggers, or trackballs, generally whileviewing on a (screen) monitor associated with a computer, web TV ornetworked system such as the Internet. For gaming users, an automatedtool as above allows assessment of whether they are “doing it right”(maintaining a proper position throughout), while joysticking,trackballing or typing. Users of keypads such as the Blackberry orvertical market devices, can view this invention on a screen/monitorassociated with a computer, web TV or networked system such as theInternet.

While this invention has been particularly shown and described withreferences to preferred embodiments thereof, it will be understood bythose skilled in the art that various changes in form and details may bemade therein without departing from the scope of the inventionencompassed by the appended claims.

1. A computer system for tracking a user's posture, comprising: anenvironmental module for determining the user's physical environment; abiomechanical module for determining at least a user's posture in theuser's physical environment; and an output module for outputting to theuser an indication of at least the user's posture relative to at least atarget correct posture.
 2. The system of claim 1, wherein theenvironmental module includes: a scene module; and a snapshot module,the scene and the snapshot modules capture a digital representation of abackground of the user's physical environment.
 3. The system of claim 1,wherein the biomechanical module includes: an active scene module forcapturing a digital representation of the user in the user's physicalenvironment; and a runtime mirror module for processing the digitalrepresentation of the user in the user's physical environment.
 4. Thesystem of claim 1, wherein the output module includes a display fordisplaying to the user the user's posture relative to a target correctposture.
 5. The system of claim 4, wherein the output module furtherincludes an audio device for outputting to the user an audio indicationof the user's posture relative to the target correct posture.
 6. Thesystem of claim 4, wherein the output is displayed in a window of amulti-window environment.
 7. The system of claim 4, wherein arepresentation of the user's posture is superimposed with the targetcorrect posture.
 8. The system of claim 1, wherein the physicalenvironment includes one of a computer workstation environment, amanufacturing environment, a gaming environment, and a keypaddingenvironment.
 9. The system of claim 1, wherein at least one digitalcamera is used in determining the user's physical environment.
 10. Thesystem of claim 1, wherein at least one digital camera is used indetermining the user's posture.
 11. A method for tracking a user'sposture, comprising computer implemented steps of: determining theuser's physical environment; determining a user's posture based on theuser's physical environment; and outputting to the user an indication ofthe user's posture relative to a target correct posture.
 12. The methodof claim 11, wherein determining the user's physical environmentincludes capturing a digital representation of a background of theuser's physical environment.
 13. The method of claim 11, whereindetermining a user's posture based on the user's physical environmentincludes: capturing a digital representation of the user in the user'sphysical environment; and processing the digital representation of theuser in the user's physical environment.
 14. The method of claim 11,wherein outputting to the user an indication of the user's postureincludes displaying to the user the user's posture relative to thetarget correct posture.
 15. The method of claim 14, wherein outputtingto the user an indication of the user's posture further includesoutputting to the user an audio indication of the user's posturerelative to a target correct posture.
 16. The system of claim 14,wherein the output is displayed in a window of a multi-windowenvironment.
 17. The method of claim 14, wherein a representation of theuser's posture is superimposed with the target correct posture.
 18. Themethod of claim 11, wherein the physical environment includes one of acomputer workstation environment, a manufacturing environment, a gamingenvironment, and a keypadding environment.
 19. The method of claim 11,wherein at least one digital camera is used in determining the user'sphysical environment.
 20. The method of claim 11, wherein at least onedigital camera is used in determining the user's posture.
 21. A systemfor tracking a user's posture, comprising: means for determining theuser's physical environment; computer means for determining a user'sposture based on the user's physical environment; and means foroutputting to the user an indication of the user's posture relative to atarget correct posture.